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APPLICATION  OF 
SYSTEMS  ENGINEERING 
TO  RAPID  PROTOTYPING 
FOR  CLOSE  AIR  SUPPORT 

f  John  M.  Colombi  and  Richard  G.  Cobb 

Twenty-first  century  military  operations  have  brought  forth 
many  new  challenges  for  the  Armed  Forces  of  the  United 
States.  One  such  challenge  is  with  new  operating  environ¬ 
ments,  where  current  systems  are  not  always  effective.  While 
it  is  desirable  to  apply  a  systems  engineering  approach  to  best 
meet  critical  user  needs,  there  may  be  a  misconception  that 
systems  engineering  requires  a  lengthy  and  detailed  process 
not  nimble  enough  for  a  rapid  prototyping  effort.  This  article 
describes  how  a  classic  systems  engineering  methodology 
was  successfully  tailored  to  the  rapid  development  of  poten¬ 
tial  material  solutions  to  meet  a  critical  operational  need.  Key 
observations  are  drawn  from  this  experience  and  formulated 
into  heuristics  for  tailoring  systems  engineering  for  future 
rapid  prototyping  efforts. 


Keywords:  Systems  Engineering,  Prototyping,  Rapid 
Product  Development,  Project  Selection,  Close  Air 
Support 
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Within  the  U.S.  Air  Force,  a  critical  need  has  emerged  for  an  added  capability 
associated  with  the  position  of  Joint  Terminal  Attack  Controller  (JTAC)— the 
Air  Force  airman  trained  to  interface  with  aircraft  to  request  and  direct  Close 
Air  Support  (CAS)  attacks:  to  quickly  pinpoint  the  location  of  friendly  ground 
forces  and  communicate  their  location  to  CAS  aircraft.  Current  operations  in 
urban  environments  have  placed  JTACs  in  very  close  proximity  to  enemy  forces 
and  reduced  the  time  to  communicate  with  CAS  assets.  This  close  proximity 
and  time  compression,  coupled  with  the  complexity  of  the  urban  terrain,  has 
made  it  difficult  for  the  JTAC  to  direct  an  air  attack  using  current  systems  and 
tactics  while  maintaining  an  acceptable  fratricide  risk.  Thus,  a  Friendly  Marking 
Device  (FMD)  that  allows  a  JTAC  to  quickly  and  accurately  identify  the  position 
of  friendly  ground  personnel  to  CAS  aircraft  has  emerged  as  a  critical  need. 


CAN  A  DEVELOPMENT  EFFORT  BE  RESPONSIVE  ENOUGH 
TO  REACT  TO  CRITICAL  NEEDS  WHILE  STILL  BENEFITING 
FROM  THE  RIGOR  OF  SYSTEMS  ENGINEERING? 


Systems  engineering  offers  a  rigorous  and  repeatable  methodology 
for  translating  a  critical  need  into  a  viable  solution  (Defense  Acquisition 
University  [DAU],  2001).  However,  the  perception  that  it  necessitates  a  lengthy 
and  detailed  process  may  contribute  to  a  misconception  that  the  benefits  of 
systems  engineering  must  be  traded  off  to  be  able  to  respond  quickly  to  critical 
user  needs.  This  perception/misconception  juxtaposes  a  key  question:  Can  a 
development  effort  be  responsive  enough  to  react  to  critical  needs  while  still 
benefiting  from  the  rigor  of  systems  engineering? 

This  article  attempts  to  answer  that  question  by  detailing  an  effort  to  tailor 
and  apply  systems  engineering  to  a  collaborative  research  project  to  rapidly 
prototype  novel  designs  for  the  FMD.  It  describes  the  methods  employed 
and  offers  key  observations  from  the  experience  as  lessons  learned.  From  the 
lessons,  heuristics  are  derived  to  guide  the  tailoring  and  application  of  systems 
engineering  to  future  rapid  prototyping  efforts. 

The  JTAC  user  identified  the  critical  need  for  a  new  way  to  mark  the 
location  of  friendly  ground  forces.  Under  the  auspices  of  the  Air  Force  Research 
Laboratory  (AFRL)  Rapid  Reaction  Program— a  program  designed  to  match 
innovative  research  initiatives  to  critical  needs— an  effort  began  aimed  at 
identifying  and  applying  technology  to  the  critical  operational  need,  and 
resulting  in  the  generation  of  a  viable  solution. 
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Method 


PROJECT  DEFINITION 

The  first  step  in  defining  the  project  was  to  assemble  a  core  project  team 
to  guide  the  development  effort.  During  this  step,  key  stakeholders  were 
identified— user/customer,  project  sponsor,  systems  engineers,  and  technology 
experts.  The  core  team  then  worked  to  understand  the  operational  need 
and,  thereby,  define  the  objective  of  the  project:  Develop,  demonstrate,  and 
transition  a  marking  solution  that  enables  a  JTAC  to  establish  a  common  point- 
of-reference  with  a  CAS  asset  such  that  the  CAS  asset  can  attack  an  intended 
target  while  avoiding  fratricide. 

Constraining  factors  such  as  cost,  schedule,  technology  maturity,  resource 
availability,  and  operational  limitations  were  clearly  identified.  Arguably, 
the  most  significant  constraint  on  the  project  was  a  compressed  schedule, 
inherent  to  the  rapid  reaction  process.  Driven  by  the  desire  to  rapidly  field  a 
prototype,  the  project  was  constrained  to  5  months.  These  constraints  became 
fundamental  elements  driving  several  key  evaluation  and  technical  focus  factors 
in  our  systems  engineering  process. 


TAILORED  APPROACH 

After  careful  consideration  of  a  variety  of  approaches,  the  classic  Vee 
model  described  in  Dennis  M.  Buede’s  (2000)  text  was  tailored  and  selected 
as  the  basis  for  this  project.  Both  the  construct  and  execution  of  the  model 
were  modified  to  accommodate  the  constraints  identified  at  the  outset.  The 
tailored  Vee  model  (Figure  1)  follows  the  general  construct  of  the  classic  Vee 
model  in  that  requirements  solicitation  and  definition  occurs  down  the  left 

FIGURE  1.  TAILORED  VEE  MODEL 
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TABLE  1.  SAMPLE  USE  CASE 


Urban  Close  Air  Support  Use  Case 

Use  Case  Name 

Name:  Urban  Close  Air  Support 

Brief  Description:  Describes  the  process  directing  a 

CAS  attack  in  an  urban  environment. 

Actors  Involved 

Joint  Tactical  Air  Controller  (JTAC):  A  certified 

servicemember  who  directs  the  action  of  aircraft 

engaged  in  close  air  support. 

•  Goal— Accurately  identify  target  and 
friendly  forces  to  CAS  aircraft. 

Close  Air  Support  (CAS)  Aircraft:  Aircraft  tasked  to 
support  close  air  support  operations. 

•  Goal— Accurately  acquire  target  and 

friendly  position. 

Air  Support  Operations  Center  (ASOC):  The  principal 
air  control  agency. 

Preconditions 

JTAC  has  communication  with  ASOC. 

JTAC  has  requested  CAS  support. 

CAS  aircraft  tasked  to  support  the  JTAC. 

CAS  has  aircraft  in  contact  with  JTAC. 

Success  Guarantee 

CAS  aircraft  provide  bombs  on  target. 

There  is  no  fratricide  of  friendly  forces. 

Collateral  damage  has  been  minimized. 

Flow  of  Events 

Main  Success  Scenario:  Sequential,  numbered  steps 
to  carry  out  the  task. 

Postconditions 

CAS  Aircraft:  Provide  bombs  on  target. 

side  (decomposition  and  definition),  design  engineering  occurs  at  the  vertex, 
and  qualification  occurs  moving  up  the  right  side.  An  important  element 
of  tailoring  as  applied  herein  involves  the  recognition  that  the  output  of  this 
tailored  Vee  model  is  not  a  validated  system  ready  for  use  in  the  field.  Rather  it 
is  an  analytically  tested  and  evaluated  prototype  that  may  be  easily  readied  for 
production  and,  ultimately,  used  in  the  field. 


PROBLEM  DEFINITION 

To  state  the  problem  in  solution-independent  terms,  the  definition  process 
began  by  exploring  the  problem  domain.  After  a  literature  search  of  typical  CAS 
processes  (Joint  Chiefs  of  Staff,  2003;  Pirnie  et  al.,  2005),  a  set  of  elicitation 
questions  was  developed  to  help  define  a  common  understanding  of  the  problem 
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FIGURE  2.  EXTERNAL  SYSTEMS  DIAGRAM 
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with  the  user.  These  questions  were  then  used  as  a  basis  for  interviewing  the 
user  representative  to  build  a  definition  of  the  problem. 

It  became  evident  the  original  problem  statement  did  not  capture  another 
perspective  that  existed— that  of  the  CAS  pilot.  To  correct  this,  experienced 
CAS  pilots  were  interviewed  in  a  similar  fashion  to  explore  their  perspective 
of  the  problem.  After  compiling  the  results  of  the  interviews,  the  problem  was 
stated  as:  The  Joint  Terminal  Attack  Controller  (JTAC)  lacks  a  covert  means  to 
quickly  and  accurately  mark  the  location  of  friendly  forces. 


OPERATIONAL  CONCEPT 

The  next  step  was  development  of  the  concept  of  operation  for  the  solution— 
the  vision  of  how  the  user  might  employ  the  resultant  device.  Borrowing  from 
software  engineering  (Larman,  2004),  the  concept  of  a  use  case  was  employed 
to  create  a  description  of  the  sequenced  actions  that  the  user  would  likely  follow 
in  employing  the  FMD  (Cockburn,  2001).  Table  1  shows  a  simplified  version  of 
the  basic  use  case  for  directing  CAS  attacks  in  an  urban  environment.  (This  is 
not  a  complete  use  case  and  is  included  for  illustration  only.) 

Buede  (2000,  p.  144)  states,  “The  single  largest  issue  in  defining  a  new 
system  is  where  to  draw  the  system’s  boundaries.”  As  the  project  progressed,  the 
value  of  defining  and  documenting  the  system  boundary  became  evident,  and 
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the  External  Systems  Diagram  shown  in  Figure  2  was  developed.  Creating  the 
External  Systems  Diagram  helped  highlight  the  key  interaction  in  the  operational 
concept— the  use  of  the  FMD  to  establish  a  common  point  of  reference  between 
the  JTAC  and  the  CAS  pilot. 


Requirements 

With  the  appropriate  data  from  the  informal  interviews  of  the  user  and  other 
stakeholders  as  guidance,  the  system  requirements  were  derived  in  detail  from 
the  operational  concept.  Once  the  initial  set  of  requirements  was  identified, 
it  was  validated  with  the  user  and  other  stakeholders.  In  addition,  the  user 


TABLE  2.  USER  REQUIREMENTS 


User  Requirements  with  Weights 

Type 

Requirements 

Weights  (1-10) 

Environmental 

Weather  Limitations 

9 

Day/Night  Limitations 

10 

Physical 

Waterproof 

8 

Shockproof 

8 

Power  Source 

8 

Weight 

10 

Size  Dimensions 

10 

Operational 

Signal  Duration 

10 

(Signal) 

Signal  Covertness 

10 

Signal  Field  of  View 

7 

Signal  Range 

10 

Accuracy  Resolution 

10 

Signal  Spectrum 

10 

System  Compromise 

2 

Unique  Signal 

2 

Signal  Delay 

10 

Operational 

Ease  of  Use 

8 

(System) 

Modification  Required 

8 

Unique  Signal  Display 

2 

Acquisition 

Long-term  Unit  Cost 

5 

(Long-term) 

Product  Feasibility 

8 

Acquisition 

Estimated  Cost 

5 

(Short-term) 

Prototype  Availability 

7 

System  Maturity 

7 
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and  other  stakeholders  provided  weights  for  each  requirement  to  determine 
their  priority.  Table  2  shows  a  sample  of  the  system  requirements  (without  the 
associated  values,  but  with  user  weights). 


OBJECTIVES  HIERARCHY 

In  making  a  decision  or  evaluation,  the  development  of  a  value  model  (in 
this  case,  an  objectives  hierarchy)  enables  the  systematic  identification  and 
application  of  user  value  to  multiple  attributes  of  a  decision.  Following  the 
approach  described  by  Ralph  L.  Keeney  (1992),  a  set  of  appropriate  objectives 
was  identified.  Attributes  to  measure  the  degree  to  which  the  objectives  are  met 
were  also  developed.  Finally,  a  hierarchy  defining  the  relative  weighting  of  the 
objectives  was  created  (Figure  3). 

The  use  case  and  user-expressed  desires  and  constraints  served  as  inputs 
into  the  development  of  the  hierarchy.  The  objectives  were  developed  by 


FIGURE  3.  SAMPLE  OBJECTIVES  HIERARCHY 
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FIGURE  4.  SAMPLE  UTILITY  CURVE 
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working  closely  with  the  user/customer.  Once  the  basic  hierarchy  had  been 
constructed,  the  user  was  solicited  for  the  relative  weightings  that  define  the 
value  or  importance  of  each  of  the  various  objectives.  Relative  weights  for 
applicable  objectives  were  also  solicited  from  the  CAS  pilots.  Utility  curves 
were  produced  based  upon  the  information  gleaned  during  the  development 
of  the  problem  definition  and  operational  concept.  Risk-neutral  utility  curves, 
also  described  in  Keeney,  were  used  in  the  assessment  of  value  for  each  of 
the  characteristics  of  the  hierarchy.  Figure  4  shows  an  example  of  the  utility 
values  for  signal  detection  range.  The  assignment  of  utility  values  and  the 
performance,  physical,  and  environmental  element  utility  curves  were  based 
upon  user  requirements. 

The  objective  hierarchy  was  continually  updated  throughout  the  FMD 
systems  engineering  process  as  candidate  technologies  matured  and  were 
tested.  It  served  as  the  primary  decision-making  tool  for  initial  candidate 
selection,  as  well  as  the  subsequent  testing  and  evaluation  to  designate 
candidates  for  transition  to  full  development. 


DEVELOP  VALIDATION/VERIFICATION  CRITERIA 

The  next  step  involved  developing  the  criteria  necessary  to  verify  the  poten¬ 
tial  solutions  against  the  derived  requirements,  and  further  validating  them 
against  the  user  need  or  mission  requirement.  The  problem  statement,  opera¬ 
tional  concept,  and  requirements  set  served  as  the  sources  for  these  criteria. 


Application  of  Systems  Engineering  to  Rapid  Prototyping  for  Close  Air  Support 


October  2009 


2  9  3 


The  basic  approach  involved  breaking  the  problem  down  into  critical 
operational  issues  (COI).  Measures  of  effectiveness  (MOE)  were  then  developed 
for  each  COI  to  help  evaluate  whether  or  not  a  particular  candidate  was  able 
to  resolve  the  issue.  MOEs  were  then  broken  down  into  specific  measures  of 
performance  (MOP)  that  could  be  measured  to  verify  the  candidate  design 
(Roedler  &  Jones,  2005;  Sproles,  2000;  Sproles,  2001).  Great  care  was  taken  to 
state  these  criteria  in  solution-independent  terms  such  that  the  evaluation  did 
not  suggest  or  favor  a  particular  type  of  solution. 


CANDIDATE  IDENTIFICATION  AND  DEVELOPMENT 

The  process  of  identifying  candidate  technologies  began  with  a  meeting 
of  the  stakeholders  to  present  the  critical  need  and  the  resulting  operational 
problem.  The  technology  experts  were  then  given  the  operational  concept 
and  the  requirements  for  the  FMD,  and  asked  to  identify  novel  technology 
candidates  to  solve  the  operational  problem.  An  initial  set  of  15  candidate 
technologies  resulted. 

This  initial  set  of  candidates  was  evaluated  for  feasibility  using  the 
objectives  hierarchy.  This  initial  evaluation  helped  to  eliminate  non-viable 
candidates.  Based  upon  this  evaluation,  the  initial  set  of  15  was  pared  down 
to  10  promising  candidates.  Over  approximately  3  months,  the  technology 
experts  worked  in  parallel  to  further  research  and  develop  their  respective 
ideas  for  solving  the  problem. 


LAB  PROTOTYPE  TESTING 

Many  of  the  decisions  to  this  point  had  been  made  based  upon  predictions, 
analytical  calculations,  and  bench  tests— analyzing  only  portions  of  the  device 
without  testing  full  functionality.  It  was,  therefore,  necessary  to  verify  the 
prototypes  through  lab  testing— testing  the  full  functionality  of  the  device  without 
subjecting  it  to  a  realistic  operational  environment.  Since  the  prototypes  were 
completed  at  different  times,  lab  testing  occurred  throughout  the  development 
period  rather  than  during  a  specific  test  period. 

To  proceed  to  the  field  demonstration,  prototypes  were  required  to  have 
been  successfully  verified  against  the  requirements  via  the  lab  testing.  The 
results  of  the  lab  tests  were  fed  back  into  the  objectives  hierarchy,  and  the 
candidate  technologies  were  again  evaluated  against  the  objectives.  As  a 
result  of  the  verification  process,  eight  prototypes  were  selected  to  proceed 
to  field  demonstration. 


OPERATIONAL  PROTOTYPE  FIELD  DEMONSTRATION 

To  properly  scope  the  demonstration,  the  team  developed  and  coordinated 
a  test  plan,  which  outlined  the  roles  and  responsibilities  of  each  participant 
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and  the  major  test  objectives.  Test  and  Evaluation  Management  guidance  is 
well  documented  (DAU,  2005).  The  test  objectives  were  derived  from  the  user 
requirements  and  MOPs  discussed  previously.  In  addition,  aircraft  flight  profile 
descriptions  were  developed,  and  a  prioritized  test  point  matrix  was  created. 
Finally,  data  requirements  were  documented  to  enable  post-flight  analysis  of 
prototype  performance. 

The  candidate  prototypes  were  taken  to  the  Nellis  Air  Force  Base  test 
range  for  the  field  demonstration.  The  evaluations  were  conducted  by  Air  Force 
operational  test  agencies  representing  both  user  communities:  the  JTAC  ground 
controllers  and  the  combat  aircrews. 


Evaluation  of  Results 

The  team  collected  and  reviewed  the  recorded  data  from  the  aircraft 
to  determine  the  maximum  detection  for  each  device  as  well  as  to  evaluate 
the  quality  of  the  detection  display  as  seen  from  the  aircraft.  JTAC  usability 


Overall,  the  FMD  project  successfully  applied 

SYSTEMS  ENGINEERING  TO  TAKE  A  CRITICAL  USER  NEED 
AND  RAPIDLY  PRODUCE  VIABLE  PROTOTYPES  THAT 
COULD  BE  TRANSITIONED  TO  PRODUCTION. 


assessments  and  aircrew  comments  were  also  gathered  and  reviewed  in  order 
to  evaluate  the  performance  of  the  prototype  devices.  While  not  a  quantitative 
measure,  the  user  assessments  of  the  prototypes  at  this  early  stage  were  deemed 
critical  as  they  would  provide  the  direction  for  the  next  phase  of  development- 
producing  the  FMD.  That  is,  once  the  basic  technology  is  proven,  it  must  still 
be  designed  to  meet  users’  expectations  for  form,  fit,  and  function.  With  this  in 
mind,  a  review  was  conducted  on  the  user  assessments  of  each  device,  noting 
favorable  characteristics  as  well  as  highlighting  key  areas  of  concern  to  be 
addressed  in  the  next  iterations  of  the  development  process. 


PRIORITIZATION  AND  SELECTION  OF  OPTIONS 

The  results  of  the  field  demonstration  were  fed  back  into  the  objective 
hierarchy.  Coupling  the  updated  ranking  from  the  objective  hierarchy  analysis 
with  engineering  judgment  and  qualitative  user  feedback,  the  team  selected  one 
candidate  technology  that  met  all  of  the  objectives  and  held  the  greatest  promise 
of  being  developed  into  a  system  capable  of  meeting  the  needs  of  the  user. 

Overall,  the  FMD  project  successfully  applied  systems  engineering  to 
take  a  critical  user  need  and  rapidly  produce  viable  prototypes  that  could 
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be  transitioned  to  production.  During  the  course  of  the  efforts,  the  systems 
engineers  gained  valuable  insight  into  the  application  of  systems  engineering 
to  rapid  prototyping.  The  remainder  of  the  article  focuses  on  key  observations. 


Key  Observations  and  Results 

In  this  section,  key  observations  are  made  about  the  FMD  project.  In 
particular,  each  section  presents  a  lesson  learned  and  briefly  describes  the 
impact  the  finding  had  on  the  project. 


UNDERSTANDING  KEY  CONSTRAINTS 

Observation:  Explicitly  stating  and  understanding  key  constraints  helped  guide  team 
decision  making  and  brought  clarity  to  choices. 

Several  key  constraints  were  established  at  the  beginning  of  the  project.  By 
stating  the  constraints  explicitly  from  the  outset,  the  entire  team  was  focused  on 
the  same  goals.  This  shared  understanding  guided  decision  making  throughout 
the  project.  In  particular,  it  made  the  choice  between  alternatives  relatively  clear 
when  conducting  trade-offs  and  candidate  evaluations. 


UNDERSTANDING  THE  LARGER  CONTEXT 

Observation:  An  understanding  of  the  larger  context  helped  in  developing  a  tailored 
systems  engineering  model  and  provided  a  long-term  framework  for  the  project. 

Part  of  tailoring  the  systems  engineering  approach  involved  understanding 
the  bigger  context  in  which  this  specific  rapid  prototyping  effort  fit.  The 
programmatic  boundary  helped  communicate  scope  to  all  the  stakeholders, 
and  helped  in  day-to-day  systems  engineering  management.  Figure  5  places 
the  modified  Vee  model  of  Figure  1  into  the  larger  context  of  a  longer-term 
development  fielding  of  future  CAS  systems  acquisitions.  In  this  context,  the 
rapid  prototyping  Vee  model  represents  the  first  increment  of  the  FMD  rapid 
fielding  effort.  This  can  also  be  viewed  as  the  first  spiral  in  the  context  of  the 
systems  engineering  spiral  model  as  shown  in  Figure  6.  This  understanding 
helped  to  modify  the  classic  Vee  model  to  one  in  which  the  end  state  was  a 
demonstrated  and  validated  FMD  prototype.  This  prototype  then  provided  both 
the  input  to  the  next  spiral  — FMD  production  design— as  well  as  a  refined  and 
validated  set  of  user  requirements  that  can  serve  as  important  inputs  for  future 
CAS  systems  acquisitions. 

In  the  spiral  development  context  (Boehm  &  Hansen,  2001),  FMD  production 
design  continues  the  spiral,  resulting  in  a  production-ready  design  to  “fill  the  gap” 
in  capability.  After  user  evaluation  and  acceptance  of  the  production  design, 
the  FMD  production  and  fielding  spiral  ensues.  A  formal  systems  acquisition 
program  for  an  advanced  FMD  was  envisioned  as  the  next  spiral. 
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FIGURE  5.  FRIENDLY  MARKING  DEVICE  (FMD)  ACQUISITION  CONTEXT 
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BORROWING  FROM  OTHER  DISCIPLINES 

Observation:  Proven  techniques  from  software  engineering  were  applicable  in  a  rapid 
hardware  prototyping  effort. 

The  field  of  software  engineering  has,  through  many  years  of  evolution, 
developed  a  very  elegant  approach  to  tame  the  complexity  and  constant 
change  of  modern  software  development.  Whereas  the  waterfall  approach 
(Royce,  1970)  treated  the  requirements  definition,  design,  and  testing  as  distinct, 
sequential  steps,  modern  approaches  such  as  the  Rational  Unified  Process 
(RUP)  (Krutchen,  2000)  emphasize  evolutionary  development  in  iterations. 
The  FMD  project  applied  key  tenets  from  the  RUP  to  the  rapid  development  of 
hardware  prototypes. 

The  sequential  waterfall  approach  presumes  that  the  requirements  for  the 
system  can  be  known  with  a  high  degree  of  certainty  from  the  outset  and  that 
those  requirements  remain  relatively  static  during  the  development  process.  In 
a  rapid  prototyping  effort,  this  is  not  very  likely  to  be  the  case,  particularly  when 
the  user  may  not  know  what  is  within  the  realm  of  the  possible  given  the  current 
state  of  the  technology  and  the  key  constraining  factors. 

The  RUP,  in  contrast,  makes  no  such  presumption  and  relies  on  short 
development  steps  with  rapid  feedback  to  adapt  the  design  as  requirements 
are  clarified.  The  FMD  project  resembled  the  RUP  in  that  it  included  an 
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FIGURE  6.  FRIENDLY  MARKING  DEVICE  (FMD)  IN  SPIRAL  CONTEXT 
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initial  exploratory  phase  much  like  an  inception  iteration.  This  phase  lasted 
approximately  4  weeks.  It  included  the  initial  meetings  with  the  user  and  the 
entire  project  team.  Accomplishments  included  creating  the  operational  concept 
(vision),  collecting  the  user’s  initial  requirements,  and  defining  the  scope  of  the 
project.  In  addition,  the  initial  technology  exploration  was  used  to  check  the 
feasibility  of  the  novel  technology  ideas.  Based  upon  initial  design  ideas  and 
performance  estimations,  the  user  was  able  to  refine  the  requirements  and  help 
eliminate  some  technology  candidates  because  of  their  size,  weight,  or  power 
consumption.  The  result  was  the  initial  list  of  ten  candidates. 

The  rest  of  the  project  (as  of  this  writing)  was  much  like  the  elaboration  phase 
of  the  RUP.  The  ten  initial  candidates  were  built  into  functioning  prototypes.  As 
the  designers  completed  various  phases  of  their  fabrication  work,  more  was 
learned  about  each  of  the  candidates.  This  new  knowledge  was  rapidly  fed  back 
into  the  process  to  further  refine  requirements  and  guide  the  project. 

Timeboxing  was  also  effective  for  the  FMD  project.  Two  candidate 
technologies  were  not  mature  enough  to  proceed  to  the  field  demonstration. 
Rather  than  slip  the  date,  those  candidates  were  excluded  from  the  field 
demonstration  with  the  intent  to  continue  their  development  and  take  them  to 
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the  field  during  a  later  iteration.  In  the  interim,  feedback  from  poor  field  results 
for  candidates  with  similar  technology  (i.e.,  employing  a  similar  type  of  emitter) 
showed  that  one  of  the  immature  candidates  would  not  be  a  viable  solution. 
That  candidate  was  eliminated,  saving  both  time  and  money. 


SELECTING  AND  USING  TOOLS 

Observation:  Selection  of  tools  suited  to  the  tailored  systems  engineering  approach 
facilitated  the  decision-making  process. 

In  making  any  decision,  the  development  of  a  value  model  enables  the 
systematic  identification  and  application  of  user  value  to  multiple  attributes  of 
a  decision.  The  FMD  rapid  development  environment  required  a  decision  tool 
that  effectively  used  the  limited  candidate  attribute  information,  preserved 
design-independent  solutions,  did  not  impose  a  large  analytical  overhead,  and 
effectively  identified  the  most  viable  alternatives. 

Within  the  framework  of  the  objectives  hierarchy,  a  “living”  multi¬ 
attribute  decision  tool  was  created  by  revisiting  the  phases  as  new  and  refined 
information  was  obtained.  In  this  way,  any  new  information,  such  as  better 
performance  estimates  or  actual  test  results,  was  quickly  fed  back  into  the 
objectives  hierarchy  to  give  a  new  snapshot  of  the  solution  space  in  terms  of  the 
stakeholders’  objectives. 

Buede  (2000)  discusses  how  the  use  of  objectives  hierarchy  can  be  used 
throughout  the  systems  design  life  cycle  to  support  trade  studies.  Another 
somewhat  unique  application  of  the  tool  was  that  the  objectives  hierarchy  was 
used  not  only  throughout  the  design  process  (down  the  left  side  of  the  Vee 
model),  but  also  as  an  analysis  tool  during  the  prototype  evaluation  process 
(up  the  right  side  of  the  Vee  model)  as  well.  The  objectives  hierarchy  provided 
a  mechanism  to  integrate  actual  prototype  test  data  with  long-term  rapid 
production  unit  attributes  such  as  projected  weight,  dimensions,  etc.,  into 
a  single,  scoreable  measure  to  compare  alternatives.  Doing  so  ensured  that 
important  production  and  usability  issues  were  considered  (via  estimates  and 
predictions)  in  the  final  candidate  selection. 


DEVELOPING  IN  PARALLEL 

Observation:  Parallel  development  helped  reduce  the  overall  risk  of  the  project. 

Managing  risk  is  part  of  any  project.  Rapid  prototyping  is,  arguably,  itself  a 
form  of  risk  management  in  that  the  aim  is  to  explore  a  solution  space.  However, 
in  the  case  of  the  FMD  project,  the  rapid  prototyping  attempted  to  respond  to 
a  critical  operational  need.  In  this  light,  there  was  significant  incentive  to  ensure 
that  some  solution  was  identified  that  would  be  acceptable  to  the  user. 

From  the  outset  of  the  project,  the  team  sought  to  reduce  the  risk  that  no 
acceptable  solution  would  be  found.  A  classic  risk  mitigation  technique  when 
dealing  with  innovative  and  often  immature  technology  is  to  pursue  multiple 
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parallel  paths  towards  the  same  goal.  This  approach  was  used  on  the  FMD 
project.  At  the  initial  evaluation,  rather  than  selecting  a  single  candidate  to 
build  and  test,  the  team  attempted  to  prototype  all  of  the  candidates  that  were 
predicted  to  meet  the  user  need  based  upon  the  estimates  and  performance 
calculations  supplied  for  the  first  iteration  of  the  objectives  hierarchy. 

Another  way  that  the  parallelism  helped  the  effort  was  that  lessons  learned 
by  one  of  the  parallel  tracks  could  be  fed  back  into  the  rest  of  the  tracks  to  help 
guide  and  refine  the  remaining  work.  For  example,  early  lab  tests  showed  that 
modulation  was  especially  helpful  in  making  a  signal  more  discernible  to  the 
observer.  This  information  was  then  incorporated  into  the  remaining  designs  to 
help  further  reduce  risk. 


MAINTAINING  RIGOR  IN  A  RAPID  REACTION  PROJECT 

Observation:  A  development  effort  can  be  responsive  to  critical  operational  needs  while 
maintaining  the  rigor  of  systems  engineering. 

Organizations  often  have  very  formalized  and  standardized  systems 
engineering  processes  for  product  development.  Within  the  DoD,  the  systems 
engineering  process  is  often  associated  with  a  series  of  documentation 
requirements  (formal  plans,  requirements,  etc.)  flowing  through  a  rather  large 
management  and  oversight  function,  coupled  with  a  very  di  recti  ve  series  of  formal 
reviews  (DAU,  2001;  Department  of  Defense,  1993).  However,  the  underlying 
principles  of  systems  engineering  are  present  in  the  DoD  process  (DeFoe, 
1993).  When  the  overhead  of  the  standard  formal  review  and  documentation 
requirements  is  reduced,  a  very  realistic  approach  to  conducting  rapid  and 
innovative  development  is  generated.  In  fact,  a  common  misperception  is  that 
the  DoD  imposes  a  specific  systems  engineering  process.  Rather,  the  Defense 
Acquisition  Guidebook  outlines  standard  industry  systems  engineering  models 
and  emphasizes  that  “models  usually  contain  guidance  for  tailoring,  which  is 
best  done  in  conjunction  with  a  risk  assessment  on  the  program  that  leads  the 
program  manager  to  determine  which  specific  processes  and  activities  are  vital 
to  the  program”  (DAU,  2009,  p.  12). 

Based  upon  the  results  of  the  FMD  project,  the  conclusion  is  drawn 
that  by  effectively  tailoring  the  application  of  classic  systems  engineering 
methodologies  to  the  problem  at  hand,  a  development  effort  can  be  responsive 
to  critical  operational  needs  while  maintaining  the  rigor  of  systems  engineering. 


HEURISTICS  DISCUSSION 

Rather  than  attempting  to  provide  a  recipe  for  tailoring  the  application  of 
systems  engineering  to  a  rapid  prototyping  effort,  this  section  presents  the 
lessons  learned  during  the  FMD  project  in  the  form  of  heuristics  that  can  help 
guide  similar  efforts  in  the  future  (Maier  &  Rechtin,  2002). 
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A  CUSTOM  APPLICATION 

Heuristic:  Tailor  the  application  of  classic  systems  engineering  practices  to  the  specific 
problem  at  hand. 

There  is  not  a  single,  approved  way  to  apply  systems  engineering  to  a  given 
type  of  project.  Each  critical  user  need  or  problem  is  unique.  While  similarities 
may  exist  across  any  set  of  problems,  each  exists  in  a  slightly  different  context 
and  has  its  own  set  of  challenges.  Therefore,  it  is  incumbent  upon  the  systems 
engineers  to  examine  these  discriminating  factors  and  apply  systems  engineering 
accordingly  to  arrive  at  a  suitable  approach.  In  particular,  the  systems  engineer: 
must  understand  the  larger  context  within  which  the  current  project  resides; 
should  look  for  similarities  in  and  borrow  from  other  projects  and  disciplines; 
and  should  select  the  appropriate  tools  for  the  job. 


Keeping  the  feedback  loop  open  and  rapid  proved 

KEY  TO  THE  DECISION  PROCESS. 
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Despite  the  fact  that  each  project  is  unique,  lessons  learned  on  similar 
projects  and  in  other  disciplines  may  prove  useful.  The  FMD  project  looked  to 
the  software  engineering  discipline  for  lessons  learned  and  for  techniques  to 
employ  in  developing  prototypes  where  time  is  short  and  requirements  are  not 
fully  known  or  understood.  Keeping  the  feedback  loop  open  and  rapid  proved 
key  to  the  decision  process. 

Having  the  right  tool  for  the  job  often  makes  a  world  of  difference  in  the 
effectiveness  of  the  effort.  The  FMD  project  needed  a  decision  tool  that  could 
take  the  rapid  feedback  and  continually  provide  an  up-to-date  snapshot  of  the 
solution  space.  The  objectives  hierarchy  was  well  suited  to  this  task.  As  test 
results  came  in  and  were  entered  into  the  tool,  a  new  snapshot  of  the  solution 
space  allowed  the  team  to  continue  to  pursue  promising  technologies  and  drop 
the  ones  that  did  not  perform  well. 


THE  TEAM  INTEGRATOR 

Heuristic:  Systems  engineers  can  integrate  the  team  by  being  the  hub  of  a  collaborative 
process. 

When  a  need  is  critical  and  time  does  not  permit  the  formation  of  a  formal 
team,  groups  may  come  together  in  an  ad  hoc  fashion  to  respond.  The  systems 
engineers  can  help  to  integrate  the  team’s  efforts  by  creating  a  collaborative 
process  and  serving  as  the  hub.  This  role  may  include  responsibilities  such  as 
creating  or  setting  up  collaboration  tools  and  serving  as  the  repository  for 
information.  In  short,  the  systems  engineer  must  treat  the  team  much  like  a 
system  of  systems  that  can  be  integrated  into  a  cohesive  whole. 
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A  USEFUL  RESULT 

Heuristic:  Manage  risk  aggressively,  but  if  no  solution  emerges ,  ensure  that  something 
beneficial  comes  from  the  effort— failure  is  not  an  option. 

Clearly  a  team  would  prefer  to  see  a  viable  solution  emerge  from  the  rapid 
prototyping  process.  Managing  the  risks  in  the  process  is  critical,  just  as  it  is 
in  nearly  any  endeavor.  However,  the  effort  should  not  be  considered  a  failure 
if  a  solution  does  not  emerge.  In  exploring  the  solution  space,  considerable 
knowledge  has  been  gained  and  requirements  are  better  understood.  All 
of  this  knowledge  can  be  fed  into  future  efforts,  allowing  them  to  benefit 
from  that  which  has  gone  before.  Therefore,  the  systems  engineers  must 
aggressively  manage  the  risks  to  increase  the  probability  that  a  solution  will 
be  found,  but  must  also  extract  the  key  lessons  and  knowledge  and  feed  them 
into  future  efforts. 


Managing  risk  requires  knowing  the  “box”  in 

WHICH  THE  PROJECT  MUST  OPERATE. 


Managing  risk  requires  knowing  the  “box”  in  which  the  project  must  operate. 
That  is,  the  team  must  understand  the  key  constraints.  In  so  constraining 
the  effort,  the  team  must  determine  what  must  be  given  up  to  remain  within 
the  box.  On  the  FMD  project,  not  modifying  aircraft  eliminated  a  significant 
portion  of  the  solution  space— the  price  for  meeting  the  schedule  and  budget. 
Understanding  this  box  helped  frame  each  decision. 


Conclusions 

At  the  beginning  of  the  article,  the  question  was  posed:  Can  a  development 
effort  be  responsive  enough  to  react  to  critical  needs  while  still  benefiting  from 
the  rigor  of  systems  engineering?  Experience  from  the  FMD  project  has  shown 
that  an  effort  can  indeed  maintain  the  rigor  of  systems  engineering,  yet  still  be 
nimble  enough  to  react  to  critical  user  needs  in  a  dynamic  environment.  While 
the  approach  taken  for  the  present  effort  will  certainly  not  work  for  every  rapid 
prototyping  effort,  the  key  observations  provide  some  overarching  lessons  to 
guide  future  efforts.  The  heuristics  provided  are  intended  to  be  a  few  more 
tools  in  the  systems  engineering  toolbox  to  aid  the  practitioner  in  applying 
systems  engineering  to  meet  emerging  critical  operational  needs  in  a  rapid 
prototyping  effort. 
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